Storage device, signal processor, image signal processor, and their methods

ABSTRACT

An image processing apparatus and an image processing method capable of performing matching processing with a small amount of calculation and highly accurately detecting a motion vector, etc., provided with a first feature extraction portion  13  for extracting a feature and spatial coordinates of a focused pixel from information of a current frame; a second feature extraction portion  14  for extracting from information of a reference frame a feature, a focused pixel, spatial coordinates of the focused pixel, spatial coordinates of vicinity region of the focused pixel, and distance information of the focused pixel with respect to the spatial coordinates; a database creation portion  15  for creating a database indicating relative relationship of the feature, the focused pixel, the spatial coordinates in the vicinity of the focused pixel and the distance information of the focused pixel with respect to the spatial coordinates; and a motion vector detection portion  16  for obtaining spatial coordinates of a shifted position by calculating by searching distance information linked to the feature from the database based on the feature extracted by the first feature extraction portion  15.

TECHNICAL FIELD

The present invention relates to a memory device, a signal processingapparatus, an image signal processing apparatus and signal processingmethods applicable to processing requiring matching, such as a motiondetection device and an object search device, etc. used, for example, ina motion image compression device, etc.

BACKGOUND ART

A matching processing of a signal, particularly, a matching processingof an image signal has a problem of generally causing a large amount ofcalculation. A motion vector detection by block matching as describedbelow is an example of using the matching processing.

Namely, in an image processing apparatus, motion detection for obtaininga motion vector indicating a motion of an image is one major techniquefor effectively performing the motion image compression. There are someproposed methods of obtaining the motion vector, but one major method isa method called block a matching algorithm.

FIG. 1 is a block diagram showing an example of a configuration of amotion detection device in an image signal processing apparatus of therelated art, wherein the block matching algorithm is applied.

The motion detection device 1 comprises frame memories 2 and 3 and amotion vector detection portion 4.

In the motion detection device 1, when an image signal is input from aninput terminal TIN, an information of one frame is stored in the framememory 2.

When an information of the next image is input, the previous (previouslyinput) information in the frame memory 2 is stored in the frame memory3, and the currently (this time) input information is stored in theframe memory 2.

Namely, the information of a current frame Fc is stored in the framememory 2, and an information of a reference frame Fr is stored in theframe memory 3.

Next, an information of the current frame Fc and the reference frame Fris sent to the vector detection portion 4 and divided to blocks in themotion vector detection portion 4 where a motion vector (Vx, Vy) isdetected and output from a terminal TOUT.

FIG. 2 is a view for explaining an outline of block matching algorithm.Below, the outline of algorithm will be explained with reference to FIG.2.

In this algorithm, a motion vector of a focused pixel Fc (x, y) in thecurrent frame Fc is obtained by calculating a differential absolutevalue sum of corresponding pixels in pixels in a reference block range(L×L) having the focused pixel Fc (x, y) at its center and pixels in thesame block range as the above block range (L×L) in a search area SR inthe reference frame Fr.

The above calculation is repeated while moving the block range extractedin the search area SR by one pixel, and a differential vector of thecenter position of a block having the smallest differential absolutevalue sum in all blocks and the focused pixel position is regarded asthe solution (motion vector).

Next, a processing procedure of detecting a motion vector of the pixelFc (x, y) in the current frame Fc will be explained in detail withreference to FIG. 3.

Step ST1:

In the step ST1, a search area SR using as a reference the same positionin the reference frame as the position (x, y) of the focused pixel isdetermined after starting processing ST0.

Step ST2:

In the step ST2, the maximum value of a calculation formula issubstituted to initialize a variable “min” storing the minimum value ofa calculation result. When assuming that one pixel is 8 bits and thenumber of pixels in a block is 16, 2⁸×16=4096 is assigned to thevariable “min”.

Step ST3:

In the step ST3, a counter variable “n” for counting blocks in a searcharea SR is initialized to be “1”.

Step ST4:

In the step ST4, a variable “sum” for being substituted a calculationresult is initialized to be “0”.

Step ST5:

In the step ST5, assuming that a range of a reference block is L×L, apixel in a certain block in the current frame is Fc (i, j), and a pixelin the k-th block in the search area SR of the reference frame Fr is Frk(i, j), calculation of a differential absolute value sum ofcorresponding pixels, that is, the formula 1 below is performed and thecalculation result is substituted for “sum”. $\begin{matrix}{{\sum\limits_{i = 1}^{L}\sum\limits_{j = 1}^{L}} = {{{{Fc}\left( {i,j} \right)} - {{Frk}\left( {i,j} \right)}}}} & (1)\end{matrix}$

Step ST6

In the step ST6, the relationship of large and small is distinguishedbetween the calculated differential absolute value sum “sum” and theminimum value “min” of the differential absolute value sum. When thecalculated differential absolute value sum “sum” is smaller, theprocedure proceeds to a step ST7, while when it is larger (includingbeing equal), the calculation result is not the minimum value, so thatthe step ST7 is skipped and the procedure proceeds to a step ST8.

Step ST7:

In the step ST7, the calculation result “sum” is updated by the minimumvalue “min”, and a counter value “n” of the block as a motion vectornumber is set.

Step ST8:

In the step ST8, if the block counter value “n” is the total number ofblocks in the search area SR, that is, the final block comes, it meansthe finish, so the procedure proceeds to a step ST10, while if it is notthe final block, the procedure proceeds to a step ST9.

Step ST9:

In the step ST9, the block counter value “n” is incremented to “n+1”,and the procedure proceeds to the step ST4 to repeat the calculation.

Step ST10:

In the step ST10, a motion vector is obtained from the center pixel and(x, y) of a block having a number stored in a motion number and outputthe same.

Since the block matching algorithm explained above repeats thecalculation of the formula (1), there is a disadvantage that an amountof the calculation becomes enormous and most of the time of imagecompression processing, such as MPEG, is spent for that.

DISCLOSURE OF THE INVENTION

An object of the present invention is to provide a memory device, asignal processing apparatus, an image processing apparatus and signalprocessing methods capable of performing matching processing, etc. onlywith a small amount of calculation and accurately detecting a motionvector, etc.

To attain the above object, a first aspect of the present invention is amemory device storing information on first data, comprising an inputmeans for receiving a first feature indicating a first feature offocused data in a second data being different from the first data; and amemory means for storing a second feature indicating a second feature ofdata in the first data at a plurality of positions corresponding tothird feature indicating the first feature of data in the first data anda feature in vicinity of the third feature; wherein the second featureas the second feature of the first data is output from a positioncorresponding to the first feature of the memory means.

A second aspect of the present invention is a memory device for storinginformation, comprising an input/output means for receiving oroutputting the information; and a storage portion for storing theinformation; wherein the storage portion at least stores a positionalinformation of focused data in a predetermined signal at a plurality ofpositions specified by a plurality of addresses corresponding to afeature of the focused data in the predetermined signal and value in thevicinity of the feature.

Preferably, the storage portion further stores a reliability indicatingassurance that the focused data takes a value of the feature or thevicinity of the feature.

A third aspect of the present invention is a memory device, comprisingan input means for receiving a first feature indicating a feature of afocused pixel in image data and coordinates of the focused pixel; and astorage means for storing coordinates of the focused pixel by making itcorrespond to the first feature; wherein the storage means furtherstores coordinates of the focused pixel by making it correspond to asecond feature indicating a feature of a pixel being different from thefirst feature.

Preferably, the storage means stores coordinates of the focused pixel atan address based on the first or second feature.

Also, the first feature is a pattern of value of pixel in the vicinityof the focused pixel.

Also, the memory device is a semiconductor device.

Preferably, the storage means further stores a reliability indicatingassurance that the focused pixel takes the first feature or the secondfeature.

Preferably, the storage means further stores a reliability indicatingassurance of storing coordinates of the focused pixel at an addressbased on the first feature or an address based on the second feature.

A fourth aspect of the present invention is a signal processingapparatus for performing matching processing by using a first signalincluding a plurality of first data and a second signal including aplurality of second data, comprising a first feature extraction meansfor extracting a feature as focused data being data at a focusedposition; and a storage means for storing positional information of thesecond data at positions specified by addresses corresponding to therespective features in the second data; wherein the storage means storespositional information of the second data at positions specified byaddresses corresponding to a plurality of features; and the positionalinformation of the second data corresponding to the focused data isobtained by reading the positional information to the second data storedin the storage means at an address corresponding to the feature of thefocused data.

Preferably, the storage means further stores a reliability indicatingassurance that the second data takes the feature corresponding to astored address as a feature of the second data.

Preferably, the storage means further stores a reliability indicatingassurance of storing positional information of the second datarespectively to the addresses.

Preferably, the reliability is a reliability between a plurality offeatures.

Also, the reliability is a reliability based on a spatial distance.

A fifth aspect of the present invention is an image signal processingapparatus for detecting a motion vector by using a first image signalincluding a plurality of first data and a second image signal includinga plurality of second data, comprising a first feature extraction meansfor extracting a feature as focused data being data at a focusedposition in the first image signal; a storage means for storingpositional information of the second data at positions specified by aplurality of addresses corresponding to the respective features of thesecond data and value in the vicinity of the feature; and a motionvector calculation means for obtaining the positional information of thesecond data corresponding to the focused data by reading positionalinformation to the second data stored in the storage means at an addresscorresponding to the feature of the focused data, and calculating amotion vector of the focused data by using the positional information ofthe focused data and the positional information of the obtained seconddata.

A sixth aspect of the present invention is a signal processing devicefor performing matching processing by using a first signal including aplurality of first data and a second signal including a plurality ofsecond data, comprising a first feature extraction means for extractinga feature as focused data being data at a focused position; and astorage means for storing positional information of the second data atpositions specified by addresses corresponding to the respectivefeatures in the second data; wherein the storage means stores positionalinformation of the second data at positions specified by addressescorresponding to a plurality of features; and the positional informationof the second data corresponding to the focused data is obtained byreading the positional information to the second data stored in thestorage means at a plurality of addresses corresponding to the featureof the focused data and value in the vicinity of the feature.

A seventh aspect of the present invention is an image signal processingdevice for detecting a motion vector by using a first image signalincluding a plurality of first data and a second image signal includinga plurality of second data, comprising a first feature extraction meansfor extracting a feature as focused data being data at a focusedposition in the first image signal; a storage means for storingpositional information of the second data at positions specified by aplurality of addresses corresponding to the respective features of thesecond data and value in the vicinity of the feature; and a motionvector calculation means for obtaining the positional information of thesecond data corresponding to the focused data by reading positionalinformation to the second data stored in the storage means at aplurality of addresses corresponding to the feature of the focused dataand value in the vicinity of the feature, and calculating a motionvector of the focused data by using the positional information of thefocused data and the positional information of the obtained second data.

Preferably, the motion vector calculation means obtains a motion vectorbased on reliability information in accordance with a spatial distanceof an image when obtaining the positional information of the seconddata.

Preferably, the motion vector calculation means obtains a motion vectorbased on reliability information in accordance with a feature whenobtaining the positional information of the second data.

Preferably, the motion vector calculation means obtains a motion vectorbased on a reliability information put together with a reliabilityinformation in accordance with a feature and a reliability informationbased on a spatial distance of an image when obtaining the positionalinformation of the second data.

An eighth aspect of the present invention is a signal processing methodfor performing matching processing by using a first signal including aplurality of first data and a second signal including a plurality ofsecond data, comprising a first step for extracting a feature as focuseddata being data of a focused position in the first signal; and a secondstep for storing positional information of the second data at positionsspecified by addresses corresponding to the respective features of thesecond data; wherein the second step stores the positional informationof the second data at positions specified by addresses corresponding toa plurality of features; and the positional information of the seconddata corresponding to the focused data is obtained by reading positionalinformation to the stored second data.

A ninth aspect of the present invention is an image signal processingmethod for detecting a motion vector by using a first image signalincluding a plurality of first data and a second image signal includinga plurality of second data, comprising a first step for extracting afeature as focused data being data at a focused position in the firstimage signal; and a second step for storing positional information ofthe second data at positions specified by addresses corresponding to therespective features of the second data and value in the vicinity of thefeature; a third step for obtaining the positional information of thesecond data corresponding to the focused data by reading the positionalinformation to the stored second data at an address corresponding to thefeature of the focused data, and calculating a motion vector of thefocused data by using the positional information of the focused data andobtained the positional information of the second data.

A tenth aspect of the present invention is a signal processing methodfor performing matching processing by using a first signal including aplurality of first data and a second signal including a plurality ofsecond data, comprising a first step for extracting a feature as focuseddata being data at a focused position in the first signal; and a secondstep for storing positional information of the second data at positionsspecified at addresses corresponding to the respective features of thesecond data; wherein the second step stores the positional informationof the second data at positions specified by addresses corresponding toa plurality of features; and the positional information of the seconddata corresponding to the focused data is obtained by reading thepositional information to the stored second data at a plurality ofaddresses corresponding to the feature of the focused data and value invicinity of the feature.

An eleventh aspect of the present invention is an image signalprocessing method for detecting a motion vector by using a first imagesignal including a plurality of first data and a second image signalincluding a plurality of second data, comprising a first step forextracting a feature as focused data being data of a focused position inthe first image signal; a second step for storing positional informationof the second data at positions specified by a plurality of addressescorresponding to the respective features of the second data and value inthe vicinity of the feature; and a third step for obtaining thepositional information of the second data corresponding to the focuseddata by reading the positional information to the second data stored inthe storage means at a plurality of addresses corresponding to thefeature of the focused data and value in the vicinity of the feature,and calculating a motion vector of the focused data by using thepositional information of the focused data and the positionalinformation of the obtained second data.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an example of a configuration of amotion detection device of the related art wherein block matchingalgorithm is applied.

FIG. 2 is a view for explaining an outline of block matching algorithm.

FIG. 3 is a flowchart for explaining a processing procedure of detectinga motion vector of a pixel Fc (x, y) in a current frame FC.

FIG. 4 is a block diagram showing an embodiment of a motion detectiondevice as a key part of an image processing apparatus according to thepresent invention.

FIG. 5 is a view for explaining the configuration of a motion memory inthe feature address system.

FIG. 6 is a view for explaining an example of using a pixel pattern as afeature.

FIG. 7 is a view for explaining a method of creating a feature database.

FIG. 8 is a view for explaining a method of creating a feature database.

FIG. 9 is a view showing an example of a structure of data to be storedin a database according to the present embodiment.

FIG. 10 is a view showing spatial coordinates in link data expressed twodimensionally.

FIG. 11 is a view showing an example of reliability in link data twodimensionally.

FIG. 12 is a view for explaining an input image and information nearfocused pixel.

FIG. 13 is a view showing a space in nx×ny dimension.

FIG. 14 is a view for explaining an example of storing at the samecoordinates in the space of nx×ny dimension in FIG. 12 since a patternof the coordinates (x0, y0) and a pixel pattern of the coordinates (x1,y1) are the same in an image one frame before.

FIG. 15 is a flowchart showing a procedure of creating a basic databaseof a database creation portion.

FIG. 16 is a flowchart showing a procedure of creating a database ofcalculating reliability based on a feature in advance and storing in thedatabase when creating the database.

FIG. 17 is a view showing a focused pixel in an image.

FIG. 18 is a view showing an image of calculation of each feature in thecase of storing reliability information based on a feature in adatabase.

FIG. 19 is a block diagram showing a configuration example of a motionvector detection portion according to the present embodiment in the caseof using information of spatial coordinates when checking reliability.

FIG. 20 is a block diagram showing a an example of configuration of amotion vector detection portion according to the present embodiment inthe case of not using information of spatial coordinates when checkingreliability.

FIG. 21 is a flowchart for explaining an operation of a motion vectordetection portion in the case of using spatial reliability of an image.

FIG. 22 is a block diagram showing another embodiment of a motiondetection device as a key part of an image processing device accordingto the present invention.

FIG. 23 is a flowchart for explaining an operation of a motion vectordetection portion in the case of evaluating a distance at the time of amotion vector based on a feature in the device in FIG. 22.

FIG. 24 is a block diagram of another example of a configuration of amotion vector detection portion according to the present embodiment.

FIG. 25 is a block diagram showing still another embodiment of a motiondetection device as a key part of an image processing device accordingto the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Below, preferred embodiments of the present invention will be explainedwith reference to the attached drawings.

FIG. 4 is a block diagram showing an embodiment of a motion detectiondevice as a key part of an image processing apparatus according to thepresent invention.

The motion detection device makes it possible to accurately estimate amotion vector with a small amount of calculation by creating a databaseindicating a correspondence relationship of a feature of a pixel andspatial coordinates of the pixel and detecting a motion vector based ondata of the database. The database (memory) is for storing spatialcoordinates, etc. of each pixel by using a feature of a pixel as anaddress. Alternately, spatial coordinates, etc. of a pixel may be linkedby using a feature as an index.

By using a database (memory) using a feature as an address, the matchingprocessing can be made easy. An outline thereof will be explained below.

This method makes it possible to accurately estimate a motion vectorwith a small amount of calculation by using a motion detection memory(hereinafter, referred to as an ME memory) for storing positioninformation by using a feature as an address and performing the matchingprocessing by using, for example, a surrounding pixel value as afeature.

The configuration and function of an ME memory wherein the featureaddress system is applied will be explained with reference to FIG. 5.

FIG. 5 is a view showing a configuration example of the ME memorywherein the feature address system is applied.

A normal memory is for storing a pixel value by using a positioninformation of a pixel as an address, while the ME memory stores forevery feature position information of pixels having the feature in flagaddresses FRGA1, 2, . . . , that is, B, C . . . in FIG. 5 successively.

One cell ME-B1 has a storage capacity of an amount of positioninformation. Also, the number of position information stored for thefeature is stored in a flag address FRGA0.

The feature is a pixel value in a certain block having a focused pixelat its center. For example, when assuming that a block range is 3×3, thevertical direction is “i”, the horizontal direction is “j”, and a pixelvalue of a position (i, j) is L(i, j), a feature in this case becomes asin the next formula (2).{L(i-1, j-1), L(i-1, j), L(i-1, j+1), L(i, j-1), L(i, j), L(i, j+1),L(i+1, j-1), L(i+1, j), L(i+1, j+1)}  (2)

However, when using a database (memory) wherein one coordinates iscorresponded to one feature (address) as such, if the feature slightlychanges due to an effect of noise, it is likely that it cannot bedetected at the corresponding coordinates.

A method below can be considered as a method of eliminating thedisadvantage.

1) A memory wherein a plurality of features (addresses) are correspondedto one coordinates is used. By judging reliability, correspondingcoordinates is determined on a reference frame Fr and a motion vector isdetected.

2) A database memory (memory) wherein a plurality of features(addresses) are corresponded to one coordinates and reliabilityinformation on reliability of the correspondence relationship is alsostored is used. A corresponding coordinates on a reference frame Fr isdetermined by judging reliability and a motion vector is detected.

Based on these methods, the present embodiment will be explained indetail.

Below, the specific configuration and function of the present motiondetection device will be explained in detail with reference to drawings.

The motion detection device 10 comprises a first frame memory 11, asecond frame memory 12, a first feature extraction potion 13, a secondfeature extraction portion 14, a database creation portion 15 and amotion vector detection portion 16.

Note that the motion vector detection portion 16 configures a motionvector calculation means according to the present invention.

Below, an outline of functions of the components of the motion detectiondevice 10 will be explained, then, data storing method of the databasecreation portion 15 and an estimation function of a motion vector of themotion vector detection portion 16 will be explained in due order.

First, an outline of functions of the components of the motion detectiondevice 10 will be explained.

The first frame memory 11 stores information one frame of an imagesignal input from an input terminal TIN.

The first frame memory 11 stores previously stored image information,that is, the information of a current frame Fc and outputs thepreviously stored image information (information of a current frame Fc)to a second frame memory 12 and the first feature extraction portion 13when next image information is input.

In the first frame memory 11, the stored information of the currentframe is stored in the second frame memory before receiving the nextframe.

The second frame memory 12 stores previous image information (forexample, one frame before) stored in the first frame memory 11 asinformation of a reference frame Fr.

In the second frame memory 12, the image information (information of thereference frame Fr) is output to the second feature extraction portion14 before the information stored in the first frame memory 11 overwritesthe second frame memory 12.

The first feature extraction portion 13 extracts a feature and spatialcoordinates of a focused pixel from image information from the firstframe memory 11, that is, information of a current frame Fc as thefocused frame and outputs to the motion vector detection portion 16 theextracted feature as a signal S13 a and the special coordinates of thefocused pixel as a signal S13 b.

The second feature extraction portion 14 extracts a feature of a focusedpixel, a feature in the vicinity of the feature of the focused pixel,spatial coordinates of the focused pixel, a feature of the focused pixelin the feature space, and distance information as a reliabilityinformation to a value in the vicinity of the feature from an inputimage information, for example, information of a reference frame Fr ofone frame before the focused frame, and outputs to the database creationportion 15 the extracted feature as a signal S14 a, the spatialcoordinates of the focused pixel as a signal S14 b, and the reliabilityinformation as a signal S14 c.

The database creation portion 15 registers the feature, the spatialcoordinates of the focused pixel and the reliability information of thefeature and spatial coordinates input as signals S14 a to S14 c by thesecond feature extraction portion 14 to the database.

The feature here may be an image pattern of, for example, 3×1 (a blockpattern made by three pixels) as shown in FIG. 6. The example in FIG. 6is a block pattern having pixel values of A, B and C.

Below, an outline of a method of creating a feature database will beexplained.

Here, in consideration of facilitating an explanation, the pattern madeby three pixels shown in FIG. 6 will be taken. Assuming that the pixelvalues A, B and C are a feature component 1, a feature component 2 and afeature component 3, respectively, and as shown in FIG. 7, a featurespace made by a feature axis 1, feature axis 2 and feature axis 3 whichare axes of the feature components is taken as an example.

The example in FIG. 7 is the case wherein a feature in the spatialcoordinates (w0, h0) becomes (x, y, z). As shown in FIG. 7, a featuredatabase is created by storing the spatial coordinates (w0, h0) in abuffer linked to a position of the spatial coordinates (x, y, z) in thefeature space.

Next, the structure of the feature database and a creation methodthereof in the feature space in the data creation portion 15 accordingto the present embodiment will be explained.

In the present embodiment, as shown in FIG. 8, the feature database notonly stores the spatial coordinates (w0, h0) in the buffer linked to theposition of the coordinates (x, y, z) in the feature space, but storesinformation on the spatial coordinates (w0, h0) in surrounding featuresthereof.

This is to improve robustness when extracting a motion vector later onand making it possible to detect corresponding coordinates even when thefeature information at the time of creating a database does notcompletely matches with feature information for actually detecting amotion vector.

In the present embodiment, other than storing the spatial coordinates(w0, h0), the coordinates (x, y, z) of the feature space and distanceinformation between features for storing the spatial coordinates (w0,h0) are also stored at the same time. This information is used asaccuracy information when matched with a feature obtained by a pluralityof spatial coordinates.

FIG. 9 is a view showing an example of a structure of data to be storedin the database.

The database in this example has the structure of storing spatialcoordinates of a pixel by using a feature extracted therefrom as anaddress and storing the reliability information of the spatialcoordinates with respect to the feature.

Surrounding values of a feature of a focused pixel are linked to spatialcoordinates of the focused pixel. In this case, the reliability isdetermined, for example, in accordance with the feature of the focusedpixel and distances between features in the feature space. For example,there is a tendency that the longer the distance in the feature space,the poorer the reliability of the focused pixel to be stored.

FIG. 10 is a view showing spatial coordinates corresponding torespective features in the feature space expressed two dimensionally,and FIG. 11 is a view showing an example of reliability of thecoordinates space two dimensionally.

In this example, when assuming that reliability of the focused pixel is“1”, the reliability of surrounding pixels becomes “0.5” in pixelsimmediately above, below, on the right and left, and the reliability ofperiphery pixels in the oblique directions becomes “0.3”.

The motion vector detection portion 16 obtains spatial coordinatesinformation that at which position in a former (or subsequent) frame onehaving the same feature existed from database information S15 of thedatabase creation portion 15 by using a feature extracted from the firstfeature extraction portion 13 as an index and obtains relativerelationship of the obtained existing position of the same feature inthe former (or subsequent) frame and spatial coordinates of an inputcurrent focused pixel, so that a motion vector is obtained (spatialcoordinates to move to is obtained).

Also, the motion vector detection portion 16 estimates a motion vectorbased on the reliability information as will be explained later on.

In this way, the motion vector detection portion 16 searches inside thedatabase by using a feature as an index and outputs spatial coordinatescorresponding thereto. This means information of at which position onehaving the same feature as that of the focused pixel existed in theformer or subsequent frame can be obtained.

Note that generally, the case of linking a plurality of spatialcoordinates to one feature can be considered. As a method of finding alink of spatial coordinates linked in only one way, a regulation that“being in a search area of a reference of a focused pixel or not?” or“being closest from the focused pixel or not?” may be used.

Next, a data storing method of the database creation portion 15,estimation function of a motion vector in the motion vector detectionportion 16, and obtaining method of reliability information will beexplained specifically.

First, a data storing method will be explained with reference to FIG. 12to FIG. 16.

FIG. 12 is a view for explaining an input image and the information inthe vicinity of a focused pixel.

In FIG. 12, (x0, y0) indicates spatial coordinates of a focused image,Nx indicates a width of a block specified in the vicinity of the focusedpixel, Ny indicates a height of the block specified in the vicinity ofthe focused pixel, and Yij indicates pixel level in the block region inthe vicinity of the focused pixel, respectively.

Here, as a general example, it is defined to express a block having aheight of Ny and a width of Nx at the spatial coordinates (x0, y0) in animage and a value of pixels in the block as Yij. An explanation will bemade by using the expression below.

The feature is not limited to a specific feature, but a feature used inthe present embodiment is assumed to be a vector made by a luminancevalue of every pixel in a block to facilitate understanding.

FIG. 13 is a view showing a space of nx×ny dimension.

FIG. 13 shows an example of the most basic case. Here, when assumingthat a luminance pattern in a block having a size Nx×Ny in the spatialcoordinates (x0, y0) in an image one frame before a focused frame is asshown in FIG. 11, values (coordinates, reliability information) of (x0,y0) corresponding to coordinates (Y00, Y01, . . . Ynxny) in the featurespace are stored in the feature database.

FIG. 14 is a view for explaining an example of storing at the samecoordinates in the space of nx×ny dimension in FIG. 13 because a patternof the coordinates (x0, y0) and a pixel pattern of the coordinates (x1,y1) are the same in an image one frame before.

In the case where the pixel pattern of a block at the spatialcoordinates (x1, y1) in an image becomes (Y00, Y01, . . . Ynxny) to bethe same pixel pattern as that at (X0, y0), since the feature is thesame, so that they are stored at the same coordinates in the featurespace.

Accordingly, values of two coordinates (x0, y0) and (x1, y1) are storedin (Y00, Y01, . . . Ynxny).

The fact that data stored at the coordinates (Y00, Y01, . . . , Ynxny)in the feature space is (x0, y0) and (x1, y1) means a pixel pattern inthe vicinity of the spatial coordinates (x0, y0) one frame before issame as a pixel pattern in the vicinity of the coordinates (x1, y1).

FIG. 15 is a flowchart showing a procedure of preparing a basic databaseof the database creation portion 15.

As shown in FIG. 15, the second feature extraction portion 14 extracts afeature of a reference frame Fr (step ST101) and supplies the same tothe database creation portion 15.

The database creation portion 15 performs an operation of storing in thedatabase corresponding spatial coordinates by using a feature as anindex before ending an image (steps ST102 and ST103).

Also, in the present embodiment, the motion vector detection portion 16estimates a motion vector based on the reliability information.

As methods of obtaining the reliability information, a method ofcalculating the reliability based on a feature in advance and holding ina database when creating the database and a method of calculating thereliability based on a feature at the time of obtaining a motion vectorcan be applied.

In those case, when calculating the reliability at the time of creatingthe database, the reliability J of each coordinates in each feature iscalculated and the reliability J is registered together with the spatialcoordinates (x0, y0) of the pixel when storing in the database.

FIG. 16 is a flowchart showing a database creation procedure ofcalculating the reliability based on a feature in advance and holding inthe database when creating the database.

As shown in FIG. 16, the second feature extraction portion 14 extracts afeature at the spatial coordinates (x0, y0) of a reference frame Fr(step ST111) and supplies the same to the database creation portion 15.

The database creation portion 15 shifts the feature in a specified range(step ST112) and calculates the reliability J of the coordinates (x0,y0) for each feature (step ST113).

Then, an operation of storing spatial coordinates (x0, y0) and thecorresponding reliability information into the database by using eachfeature as an index until an image ends is performed (steps ST114 andST115).

Next, the motion vector estimation based on the reliability informationin the motion vector detection portion 16 will be explained.

Here, an explanation will be made on three forms: motion vectorestimation using a reliability information based on a spatial distanceof an image, a motion vector estimation using the reliabilityinformation based on a feature of an image, and a motion vectorestimation using a reliability information wherein a feature of an imageand a reliability information based on partial coordinates of the imageare put together.

First, the motion vector estimation using the reliability informationbased on a spatial distance of an image will be explained.

A method of obtaining a motion vector when a feature database is createdas above will be as explained below.

The case of obtaining a motion vector by using the reliabilityinformation based on a spatial distance of an image can be obtained asbelow.Evaluation Value J=ΔL   (3)

The ΔL expresses the spatial distance of an image and, for example, adistance L between (x0, y0) and (x1, y1) can be expressed as below.ΔL=|(x 1, y 1)−(x 0, y 1)|  (4)

As shown in FIG. 17, a motion vector at coordinates (x′, y′) of #n+1frame can be obtained as below.

First, a value of a vicinity pixel at the coordinates (x′, y′) in the#n+1 frame is obtained.

Next, by using a pixel pattern obtained from a feature database of thedatabase creation portion 15, data stored there is obtained.

When data stored in the database is (x0, y0) and (x1, y1), that is,there are a plurality of possible coordinates, one having the minimumreliability based on a spatial distance in an image is regarded as amotion vector.

Namely, for example, if the reliability based on a spatial distance ofthe image is an absolute value of a spatial distance, a condition of theformula 5 below,|(x′, y′)−(x 0, y 0)|>|(x′, y′)−(x 1, y 1)|  (5)that is, when a condition of the next formula 6 is satisfied, theformula 7 is the solution of the motion vector, while when notsatisfied, the formula 8 is the solution of the motion vector.ΔL0>ΔL1   (6){(x′, y′)−(x0, y0)}  (7){(x′, y′)−(x1, y1)}  (8)

The above is algorithm of basic motion vector detection.

Next, the case of introducing reliability as a method of obtainingpossible coordinates will be explained.

Namely, the motion vector estimation using the reliability informationbased on a feature of an image will be explained.

In the example explained above, the reliability was calculated based ona spatial distance of an image, but a reliability regarding a feature isnewly introduced here.

The reliability based on a feature that will be introduced below isdefined as an amount expressing how much a feature T to be evaluated isdeviated with respect to a reference feature T0.

The case of obtaining a motion vector using the reliability informationbased on a feature can be obtained as below.Evaluation Value J=ΔT   (9)

The ΔT expresses a difference amount with respect to a reference featurein a feature space, and a distance between features is an examplethereof. In the following explanation, the distance between features istaken as an example.

Here, when making the feature T correspond to the above specific exampleabove, T0=(Y00, Y01, . . . YnxYny), etc. is one example thereof.

At this time, when T0, T1=(Y00′, Y01′, . . . YnxYny′), T2=(Y00″, Y01″, .. . YnxYny″), a method of calculating possible coordinates becomes asbelow.

A motion vector at coordinates (x′, y′) of #n+1 frame can be obtained asbelow.

First, a value of a vicinity pixel at coordinates (x′, y′) in the #n+1frame is obtained.

Next, by using a pixel pattern obtained from the feature database of thedatabase creation portion 15, data stored there is obtained.

When data stored in the database is (x0, y0) and (x1, y1), that is, whenthere are a plurality of candidate coordinates, one having minimumreliability based on an error amount between features (here, ΔT1 andΔT0) is regarded as a motion vector. Namely, when a condition of theformula 10 below is satisfied, the formula 11 is the solution of themotion vector, while when not satisfied, the formula 12 is the solutionof the motion vector.ΔT1>ΔT0   (10){(x′, y′)−(x0, y0)}  (11){(x′, y′)−(x1, y1)}  (12)

Next, the motion vector estimation using the reliability informationwherein a feature of an image and the reliability information based onspatial coordinates of the image are put together will be explained.

A method of creating integrated reliability as the new reliabilityinformation wherein a feature of an image and reliability informationbased on spatial coordinates of the image are put together is as below.“w0” and “w1” are coefficients of weighting.Evaluation Value J=ΔI=w0×ΔL+w1×ΔT   (13)A motion vector at coordinates (x′, y′) of the #n+1 frame can beobtained as below.

First, a value of a vicinity pixel of the coordinates (x′, y′) in the#n+1 frame is obtained.

Next, by using a pixel pattern obtained from the feature database of thedatabase creation portion 15, data stored there is obtained.

When data stored in the database is (x0, y0) and (x1, y1), that is,there are a plurality of candidate coordinates, one having minimumreliability based on an error amount in integrated reliability isregarded as a motion vector. Namely, when assuming that the reliabilitybased on a spatial distance of an image is, for example, an absolutevalue of the spatial distance, if a condition of the next formula 14 issatisfied, the formula 15 is the solution of the motion vector, whilewhen not satisfied, the formula 16 is the solution of the motion vector.ΔI1>ΔI0   (14){(x′, y′)−(x0, y0)}  (15){(x′, y′)−(x1, y1)}  (16)

Conditions of the following formula 17 are $\begin{matrix}\begin{matrix}{{Y\quad 00^{\prime}} < {Y\quad 00} < {Y\quad 00^{''}}} \\{{Y\quad 01^{\prime}} < {Y\quad 01} < {Y\quad 01^{''}}} \\\vdots \\{{Ynxny}^{\prime} < {Ynxny} < {Ynxny}^{''}}\end{matrix} & (17)\end{matrix}$

Namely, it expresses that when the coordinates stored at coordinates(Y00, Y01, . . . , Ynxny) is (x0, y0) and the coordinates stored atcoordinates (Y00+1, Y01, . . . , Ynxny) is (x1, y1), if a condition ofthe next formula 18 is satisfied, a possible motion vector becomes (x1,y1).|(x′, y′)−(x 0, y 0)|>|(x′, y′)−(x 1, y 1)|  (18)

Note that when the coordinates stored at coordinates (Y00+1, Y01, . . ., Ynxny) is (x2, y2), even if a condition of the formula 19 below issatisfied, in the case where a distance of (x2, y2) and (x0, y0) islong, it is not realistic to regard (x2, y2) as the possible motionvector.|(x′, y′)−(x 0, y 0)|>|(x′, y′)−(x 2, y 2)|  (19)

Thus, in the present embodiment, the above problem is solved by givingan upper limit to the motion vector value in addition to an evaluationonly by an image pattern.

Next, a method of obtaining the reliability information will beexplained.

A motion vector estimation method based on the reliability was explainedso far. In this method, the reliability based on a spatial feature of animage can be obtained only at the time of obtaining a motion vector,while the reliability based on a feature can be obtained also at thetime of creating a database.

Accordingly, a method of obtaining the reliability based on a featurewill be explained with two specific examples.

A first method is to calculate the reliability based on a feature inadvance at the time of creating a database and to hold in the database,and a second method is to calculate the reliability based on a featureat the time of obtaining a motion vector.

Note that a method of calculating the reliability based on a feature inadvance at the time of creating a database and holding in the databasewas already explained, so that the detail will be omitted here.

Note that FIG. 18 is a view showing an image of calculating each featurein the case of storing the reliability information based on a feature inthe database.

This example indicates an image of storing in the database a feature T1at the coordinates (x1, y1), a feature T2 at the coordinates (x2, y2), afeature TN at coordinates (xN, yN), each coordinates and reliability f.The coordinates (x1, y1) will be considered. A feature of thecoordinates (x1, y1) is T1. Here, the feature is shifted exactly by Δti(i=0, 1, 2, , n), it is made to associate with each feature T1+Δti, andthe coordinates (x1, y1) and the reliability f with respect to eachfeature at coordinates (x1, y1) is stored.

Here, a method of calculating the reliability based on a feature at thetime of obtaining a motion vector will be explained.

A motion vector at the coordinates (x′, y′) of the #n+1 can be obtainedas below.

First, a value of a vicinity pixel of the coordinates (x′, y′) in the#n+1 frame is obtained.

Next, by using a pixel pattern obtained from a feature database of thedatabase creation portion 15, data stored there is obtained.

By changing the feature to a certain extent, the reliability based on afeature in accordance with a change amount of the feature is calculatedfor each feature.

Then, in a search range in a feature space, one having the smallestreliability based on a difference amount between total evaluation valuesJ defined above is regarded as a motion vector. Namely, when assumingthat the smaller the total evaluation value, the higher the reliability,if a condition of the formula 20 is satisfied, the formula 21 is thesolution of the motion vector, while when not satisfied, the formula 22is the solution of the motion vector.ΔT1>ΔT0   (20){(x′, y′)−(x0, y0)}  (21){(x′, y′)−(x1, y1)}  (22)

FIG. 19 and FIG. 20 are block diagrams showing an example of a specificconfiguration of the motion vector detection portion 16. FIG. 19 showsthe case of also using information of spatial coordinates for thereliability checking, and FIG. 20 shows the case of not usinginformation of spatial coordinates for the reliability checking.

The motion vector detection portion 16 comprises, as shown in FIG. 19and FIG. 20, a corresponding point candidate selection portion 161, areliability checking portion 162 and a motion vector determinationportion 163.

The corresponding point candidate selection portion 161 performssearching (matching) for selecting candidates in a range of an inputfeature of T±ΔT when the reliability information is stored in thedatabase creation portion 15.

The corresponding point candidate selection portion 161 performsmatching with an input feature itself and calculates the reliabilitywhen the reliability information is not stored in the database creationportion 15.

The reliability checking portion 162 checks whether the above eachconditions of reliability is satisfied or not and supplies the checkresult to the motion vector determination portion 163.

The motion vector determination portion 163 obtains a solution of amotion vector based on the reliability check result.

FIG. 21 is a flowchart for explaining an operation of the motion vectordetection portion 16 when a plurality of features are corresponded toone coordinates. In this example, the reliability is calculated in themotion vector detection portion 16.

As shown in FIG. 21, a first feature extraction portion 13 extracts afeature of a current frame Fc (step ST201) and supplies to the motionvector detection portion 16.

In the motion vector detection portion 16, the corresponding pointcandidate selection portion 161 searches a feature which matches withdata stored in the database in a feature space (step ST202).

Then, when a feature to be matched is found in the database (stepST203), information linked to the detected feature, for example, thespatial coordinates is extracted (step ST204).

When processing in the step ST204 is finished, whether there are aplurality of candidate coordinates or not is judged (step ST205).

When there are a plurality of candidate coordinates, the reliability iscalculated (step ST206), and the reliability checking portion 162outputs coordinates having the highest reliability to the motion vectordetermination portion 163 (step ST207).

Then, in the motion vector determination portion 163, a motion vector isobtained based on a predetermined condition (step ST208).

Note that in the step ST203, when a feature to be matched is not foundin the database, it is judged that the motion vector is uncertain (stepST209).

Also, in the step ST205, when it is judged that there are not aplurality of candidate coordinates but one, the corresponding spatialcoordinates is output to the motion vector determination portion 163 viathe reliability checking portion 162 (step ST210) and the procedureproceeds to the processing in the step ST208.

The above is the case of using a memory storing spatial coordinates sothat a plurality of features (addresses) correspond to one spatialcoordinates. Other than the above, the perfect motion vector detectioncan be performed also when using a memory storing spatial coordinates sothat one feature (address) corresponds to one spatial coordinates. Themethod will be explained below.

A motion detection device 10A in this case is shown in FIG. 22.

What the motion detection device 10A differs from the motion detectiondevice 10 in FIG. 4 is a feature extraction portion 14A, a databasecreation portion 15A and a motion vector detection portion 16A.

The feature extraction portion 14A does not output the reliabilityinformation but outputs the spatial coordinates and the feature to thedatabase creation portion 15A.

The database creation portion 15A stores the input spatial coordinatesat an address corresponding to the input feature in the database(memory).

The motion vector detection portion 16A changes the input feature. Then,by using the input feature and the changed feature, address spatialcoordinates corresponding to these features are obtained from thedatabase creation portion 15A. Then, the reliability judgment isperformed on the obtained spatial coordinates, so that a motion vectoris detected.

FIG. 23 is a flowchart for explaining an operation of the motion vectordetection portion 16A in this case.

As shown in FIG. 23, the first feature extraction portion 13 extracts afeature of the current frame Fc (step ST301) and supplies the same tothe motion vector detection portion 16.

In the motion vector detection portion 16A, the corresponding pointcandidate selection portion 161 searches based on a plurality ofcandidate features corresponding to the feature (step ST302).

Also, the corresponding point candidate selection portion 161 searches afeature to be matched with data stored in the database in the featurespace (step ST303).

When a feature to be matched is found in the database (step ST304),information linked to the detected feature, for example, the spatialcoordinates is extracted (step ST305).

When processing in the step ST305 is finished, whether there are aplurality of candidate coordinates or not is judged (step ST306).

When there are a plurality of candidate coordinates, the reliability iscalculated (step ST307), and the reliability check portion 162 outputscoordinates with the highest reliability to the motion vectordetermination portion 163 (step ST308).

Then, in the motion vector determination portion 163, a motion vector isobtained based on a predetermined condition (step ST309).

Note that in the step ST304, when a feature to be matched is not foundin the database, it is judged that the motion vector is uncertain (stepST310).

Also, in the step ST305, when it is judged that there is not a pluralityof candidate coordinates but one, the corresponding spatial coordinatesis output to the motion vector determination portion 163 via thereliability checking portion 162 (step ST311).

Further, FIG. 24 is a view showing another example of a configuration ofthe motion vector detection portion 16A.

The motion vector detection portion 16A comprises a read portion 164 anda determination portion 165.

The read portion 164 changes a feature based on the feature information,reads spatial coordinates of a plurality of features (addresses) fromthe database (memory) and sends to the determination portion 165. Atthis time, if necessary, the reliability of the feature is obtained andsent to the determination portion 165. Also, the reliability of thefeature is calculated in the read portion 164.

Also, FIG. 25 is a view showing still another example of a configurationof the motion detection device.

This is an example of a motion detection device using a database(memory) storing one spatial coordinates at addresses corresponding to aplurality of features. The database (memory) does not store thereliability information.

What the motion detection device 10B differs from the motion detectiondevice 10 in FIG. 4 is a feature extraction portion 14B, a databasecreation portion 15B and a motion vector detection portion 16B.

The feature extraction portion 14A extracts a feature, changes thefeature exactly by ±Δ and outputs the changed plurality of spatialcoordinates to the database creation portion 15B.

The database creation portion 15B stores the input spatial coordinatesat an address corresponding to the input feature in the database(memory).

The motion vector detection portion 16B obtains address spatialcoordinates corresponding to the input feature from the databasecreation portion 15B, performs a reliability judgment on the obtainedspatial coordinates and detects a motion vector.

When assuming that the motion vector detection portion 16B has the sameconfiguration as that in FIG. 24, for example, the read portion 164reads spatial coordinates (reliability information) of a feature(address) from the database (memory) and sends to the determinationportion 165. Also, at that time, if necessary, the reliability of thefeature is also obtained and sent to the determination portion 165.Also, the reliability of the feature is calculated in the read portion164.

Finally, an operation of the motion detection device in FIG. 4 will beexplained.

A first frame memory 11 stores one frame of information in an imagesignal input from the input terminal TIN.

In the second frame memory 12, the previous image information (forexample, one frame before) stored in the first frame memory 11 is storedas information of the reference frame Fr.

In the second frame memory 12, the image information (information of thereference frame Fr) is output to the second feature extraction portion14 before the second frame memory 12 is overwritten by informationstored in the first memory 11.

In the second feature extraction portion 14, the feature, spatialcoordinates of a focused pixel and the vicinity of the focused pixel,and spatial distance information between the spatial coordinates and thefocused pixel are extracted from the input image information, forexample, from information of the reference frame Fr of one frame beforea focused frame.

Then, the extracted feature is output as a signal S14 a, the spatialcoordinates of the focused pixel and the vicinity thereof is output as asignal S14 b, and the spatial distance information between the spatialcoordinates and the focused pixel is output as a signal S14 c to thedatabase creation portion 15.

In the database creation portion 15, linkage of the input feature,spatial coordinates of the focused pixel and the vicinity thereof inputas signals S14 a to S14 c is registered to a database by the secondfeature extraction portion 14.

Then, for example, an image input at the next timing is once stored inthe first frame memory 11 and then supplied to the first featureextraction portion 13.

In the first feature extraction portion 13, the feature and spatialcoordinates of a focused pixel are extracted from image information fromthe first frame memory 11, that is, the feature from information of thecurrent frame Fc as the focused frame. Then, the extracted feature isoutput as a signal S13 a and the spatial coordinates of the focusedpixel is output as a signal S13 b to the motion vector detection portion16.

In the motion vector detection portion 16, by using the featureextracted in the first feature extraction portion 13 as an index, thespatial coordinates information indicating at which position one havingthe same feature existed in a former (subsequent) frame is obtained fromthe database information S15 of the database creation portion 15.

By obtaining relative relationship of an existing position of the samefeature in the former (subsequent) frame obtained here and the spatialcoordinates of the input current focused pixel, a motion vector isobtained.

In the above explanation, the spatial coordinates of the focused pixelwas stored respectively in addresses corresponding to the feature of thefocused pixel and the feature in the vicinity of the feature of thefocused pixel. However, by storing the spatial coordinates of thefocused pixel and the spatial coordinates in the vicinity of the focusedpixel at an address corresponding to the feature of the focused pixel,an idea of associating one spatial coordinates to a plurality offeatures may be also realized.

In the present embodiment, a memory device corresponding to motionvector detection was explained. Therefore, a memory device for storingspatial coordinates by using each feature as an address was taken as anexample.

However, the present invention can be applied also to a system forperforming matching other than motion vector detection. Namely, by usingas an address a first feature indicating a feature A of data, it may beconfigured to store a second feature indicating a feature B of the data.The feature A and feature B may be suitably set or changed in accordancewith an object of the system/device for performing matching. Forexample, in the present embodiment, an explanation was made assumingthat the feature A was a pixel value pattern/ADRC code and the feature Bwas coordinates, but other features may be also used.

As explained above, according to the present embodiment, since a firstfeature extraction portion 13 for extracting a feature and spatialcoordinates of a focused pixel from information of a current frame Fc asa focused frame from the first frame memory 11, and outputting theextracted feature as a signal S13 a and the spatial coordinates of thefocused pixel as a signal S13 b; a second feature extraction portion 14for extracting a feature, spatial coordinates of the focused pixel andthe vicinity thereof and spatial distance information between thespatial coordinates and the focused pixel from information of areference frame Fr of one frame before a focused frame, and outputtingthe extracted feature as a signal S14 a, the spatial coordinates of thefocused pixel and the vicinity of the focused pixel as a signal S14 band spatial distance information between the spatial coordinates and thefocused pixel as a signal S14 c; a database creation portion 15 forregistering to a database linkage of the feature and spatial coordinatesof the focused pixel and the vicinity thereof input as the signals S14 ato S14 c from the second feature extraction portion 14; and a motionvector detection portion 16 for obtaining a spatial coordinatesinformation indicating at which point one having the same featureexisted in a former (subsequent) frame from database information S15 ofthe database creation portion 15 by using the feature extracted in thefirst feature extraction portion 13 as an index, and obtaining a motionvector by obtaining relative relationship of an existing position of thesame feature in the former (subsequent) frame obtained here and thespatial coordinates of the input current focused pixel; effects belowcan be obtained.

Namely, in the present embodiment, the spatial pattern information in ablock area is regarded as a feature and distance calculation comparisonis made only for the number of candidates, so that there is an advantagethat highly accurate motion vector detection becomes possible with asmaller amount of calculation than that in a method of the related art.

INDUSTRIAL APPLICABILITY

According to an image processing apparatus and an image processingmethod of the present invention, highly accurate motion vector detectionbecomes possible, accordingly, it can be applied to processing requiringmatching as in a motion detection device and an object search device,etc. used for a motion image compression apparatus, etc.

EXPLANATION OF REFERENCES

-   10, 10A, 10B . . . motion detection device-   11 . . . first frame memory-   12 . . . second frame memory-   13 . . . first feature extraction portion-   14, 14A, 14B, . . . second feature extraction portion-   15, 15A, 15B, . . . database creation portion-   16, 16A, 16B, . . . motion vector detection portion

1. A memory device for storing information on first data, characterizedin that: an input means for receiving a first feature indicating a firstfeature of focused data in a second data being different from said firstdata; and a memory means for storing a second feature indicating asecond feature of data in said first data at a plurality of positionscorresponding to third feature indicating said first feature of the datain said first data and a feature in vicinity of said third feature;wherein said second feature as said second feature of said first data isoutput from a position corresponding to said first feature of saidmemory means.
 2. A memory device for storing information, comprising: aninput/output means for receiving or outputting said information; and astorage portion for storing said information; wherein said storageportion at least stores a positional information of focused data in apredetermined signal at a plurality of positions specified by aplurality of addresses corresponding to a feature of the focused data inthe predetermined signal and value in the vicinity of said feature.
 3. Amemory device as set forth in claim 2, characterized in that saidstorage portion further stores a reliability indicating assurance thatsaid focused data takes a value of said feature or the vicinity of saidfeature.
 4. A memory device, characterized by comprising: an input meansfor receiving a first feature indicating a feature of a focused pixel inimage data and coordinates of said focused pixel; and a storage meansfor storing coordinates of said focused pixel by making it correspond tosaid first feature; wherein said storage means further storescoordinates of said focused pixel by making it correspond to a secondfeature indicating a feature of a pixel being different from said firstfeature.
 5. A memory device as set forth in claim 4, characterized inthat said storage means stores coordinates of said focused pixel at anaddress based on said first or second feature.
 6. A memory device as setforth in claim 4, characterized in that said first feature is a patternof value of pixel in the vicinity of said focused pixel.
 7. A memorydevice as set forth in claim 4, characterized in that said memory deviceis a semiconductor device.
 8. A memory device as set forth in claim 4,characterized in that said storage means further stores a reliabilityindicating assurance that said focused pixel takes said first feature orsaid second feature.
 9. A memory device as set forth in claim 4,characterized in that said storage means further stores a reliabilityindicating assurance of storing coordinates of said focused pixel at anaddress based on said first feature or an address based on said secondfeature.
 10. A signal processing apparatus for performing matchingprocessing by using a first signal including a plurality of first dataand a second signal including a plurality of second data, characterizedby comprising: a first feature extraction means for extracting a featureas focused data being data at a focused position; and a storage meansfor storing positional information of said second data at positionsspecified by addresses corresponding to said respective features in saidsecond data; wherein: said storage means stores positional informationof said second data at positions specified by addresses corresponding toa plurality of features; and the positional information of said seconddata corresponding to said focused data is obtained by reading thepositional information to said second data stored in said storage meansat an address corresponding to said feature of said focused data.
 11. Asignal processing apparatus as set forth in claim 10, characterized inthat said storage means further stores a reliability indicatingassurance that said second data takes said feature corresponding to astored address as a feature of said second data.
 12. A signal processingapparatus as set forth in claim 10, characterized in that said storagemeans further stores a reliability indicating assurance of storingpositional information of said second data respectively to saidaddresses.
 13. A signal processing apparatus as set forth in claim 10,characterized in that said reliability is a reliability between aplurality of features.
 14. A signal processing apparatus as set forth inclaim 10, characterized in that said reliability is a reliability basedon a spatial distance.
 15. An image signal processing apparatus fordetecting a motion vector by using a first image signal including aplurality of first data and a second image signal including a pluralityof second data, characterized by comprising: a first feature extractionmeans for extracting a feature as focused data being data at a focusedposition in said first image signal; a storage means for storingpositional information of said second data at positions specified by aplurality of addresses corresponding to said respective features of saidsecond data and value in the vicinity of said feature; and a motionvector calculation means for obtaining the positional information ofsaid second data corresponding to said focused data by reading thepositional information to said second data stored in said storage meansat an address corresponding to said feature of said focused data, andcalculating a motion vector of said focused data by using the positionalinformation of said focused data and the positional information of saidobtained second data.
 16. An image signal processing apparatus as setforth in claim 15, characterized in that said motion vector calculationmeans obtains a motion vector based on reliability information inaccordance with a spatial distance when obtaining the positionalinformation of said second data.
 17. An image signal processing deviceas set forth in claim 15, characterized in that said motion vectorcalculation means obtains a motion vector based on reliabilityinformation in accordance with a feature when obtaining the positionalinformation of said second data.
 18. An image signal processing deviceas set forth in claim 15, characterized in that said motion vectorcalculation means obtains a motion vector based on a reliabilityinformation put together with a reliability information in accordancewith a feature and a reliability information in accordance with aspatial distance when obtaining the positional information of saidsecond data.
 19. A signal processing apparatus for performing matchingprocessing by using a first signal including a plurality of first dataand a second signal including a plurality of second data, characterizedby comprising: a first feature extraction means for extracting a featureas focused data being data at a focused position; and a storage meansfor storing positional information of said second data at positionsspecified by addresses corresponding to said respective features in saidsecond data; wherein: said storage means stores the positionalinformation of said second data at positions specified by addressescorresponding to a plurality of features; and the positional informationof said second data corresponding to said focused data is obtained byreading the positional information to said second data stored in saidstorage means at a plurality of addresses corresponding to said featureof said focused data and value in the vicinity of feature.
 20. An imagesignal processing apparatus for detecting a motion vector by using afirst image signal including a plurality of first data and a secondimage signal including a plurality of second data, characterized bycomprising: a first feature extraction means for extracting a feature asfocused data being data at a focused position in said first imagesignal; a storage means for storing positional information of saidsecond data at positions specified by a plurality of addressescorresponding to said respective features of said second data and valuein the vicinity of said feature; and a motion vector calculation meansfor obtaining the positional information of said second datacorresponding to said focused data by reading the positional informationto said second data stored in said storage means at a plurality ofaddresses corresponding to said feature of said focused data and valuein the vicinity of said feature, and calculating a motion vector of saidfocused data by using the positional information of said focused dataand the positional information of said obtained second data.
 21. Animage signal processing apparatus as set forth in claim 20,characterized in that said motion vector calculation means obtains amotion vector based on reliability information in accordance with aspatial distance of an image when obtaining the positional informationof said second data.
 22. An image signal processing apparatus as setforth in claim 20, characterized in that said motion vector calculationmeans obtains a motion vector based on reliability information inaccordance with a feature when obtaining the positional information ofsaid second data.
 23. An image signal processing apparatus as set forthin claim 20, characterized in that said motion vector calculation meansobtains a motion vector based on a reliability information put togetherwith a reliability information in accordance with a feature and areliability information in accordance with a spatial distance of animage when obtaining the positional information of said second data. 24.A signal processing method for performing matching processing by using afirst signal including a plurality of first data and a second signalincluding a plurality of second data, characterized by comprising: afirst step for extracting a feature as focused data being data of afocused position in said first signal; and a second step for storingpositional information of said second data at positions specified byaddresses corresponding to said respective features of said second data;wherein said second step stores the positional information of saidsecond data at positions specified by addresses corresponding to aplurality of features; and the positional information of said seconddata corresponding to said focused data is obtained by reading thepositional information to said stored second data.
 25. A signalprocessing method as set forth in claim 24, characterized in that saidsecond step further stores a reliability indicating assurance that saidsecond data takes said feature corresponding to a stored address as afeature of said second data.
 26. A signal processing method as set forthin claim 24, characterized in that said second step further stores areliability indicating assurance of storing positional information ofsaid second data respectively to said addresses.
 27. A signal processingmethod as set forth in claim 24, characterized in that said reliabilityis a reliability between a plurality of features.
 28. A signalprocessing method as set forth in claim 24, characterized in that saidreliability is a reliability based on a spatial distance.
 29. An imagesignal processing method for detecting a motion vector by using a firstimage signal including a plurality of first data and a second imagesignal including a plurality of second data, characterized bycomprising: a first step for extracting a feature as focused data beingdata at a focused position in said first image signal; and a second stepfor storing positional information of said second data at positionsspecified by addresses corresponding to said respective features of saidsecond data and value in the vicinity of said feature; a third step forobtaining the positional information of said second data correspondingto said focused data by reading the positional information to saidstored second data at an address corresponding to said feature of saidfocused data, and calculating a motion vector of said focused data byusing the positional information of said focused data and the obtainedpositional information of said second data.
 30. An image signalprocessing method as set forth in claim 29, characterized in that saidthird step obtains a motion vector based on reliability information inaccordance with a spatial distance when obtaining positional informationof said second data.
 31. An image signal processing method as set forthin claim 29, characterized in that said third step obtains a motionvector based on reliability information in accordance with a featurewhen obtaining positional information of said second data.
 32. An imagesignal processing method as set forth in claim 29, characterized in thatsaid third step obtains a motion vector based on a reliabilityinformation put together with a reliability information in accordancewith a feature and a reliability information in accordance with aspatial distance when obtaining positional information of said seconddata.
 33. A signal processing method for performing matching processingby using a first signal including a plurality of first data and a secondsignal including a plurality of second data, characterized bycomprising: a first step for extracting a feature as focused data asdata at a focused position in said first signal; and a second step forstoring positional information of said second data at positionsspecified at addresses corresponding to said respective features of saidsecond data; wherein said second step stores the positional informationof said second data at positions specified by addresses corresponding toa plurality of features; and the positional information of said seconddata corresponding to said focused data is obtained by reading thepositional information to said stored second data at a plurality ofaddresses corresponding to said feature of said focused data and valuein the vicinity of said feature.
 34. An image signal processing methodfor detecting a motion vector by using a first image signal including aplurality of first data and a second image signal including a pluralityof second data, characterized by comprising: a first step for extractinga feature as focused data as data of a focused position in said firstimage signal; a second step for storing positional information of saidsecond data at positions specified by a plurality of addressescorresponding to said respective features of said second data and valuein the vicinity of said feature; and a third step for obtaining thepositional information of said second data corresponding to said focuseddata by reading the positional information to said second data stored insaid storage means at a plurality of addresses corresponding to saidfeature of said focused data and value in the vicinity of said feature,and calculating a motion vector of said focused data by using thepositional information of said focused data and the positionalinformation of said obtained second data.
 35. An image signal processingmethod as set forth in claim 34, characterized in that said third stepobtains a motion vector based on reliability information in accordancewith a spatial distance of an image when obtaining positionalinformation of said second data.
 36. An image signal processing methodas set forth in claim 34, characterized in that said third step obtainsa motion vector based on reliability information in accordance with afeature when obtaining positional information of said second data. 37.An image signal processing method as set forth in claim 34,characterized in that said third step obtains a motion vector based on areliability information put together with a reliability information inaccordance with a feature and a reliability information in accordancewith a spatial distance of an image when obtaining positionalinformation of said second data.